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Abstract. This paper reports on the use of can-do statements (CDS) for the development 
of learning materials to prepare Japanese university students for the Test of English 
for International Communication (TOEIC). These learning materials have been made 
available on ATR CALL BRIX (http://www.atr-lt.jp/products/brix/index.html), a www- 
based courseware with a learning management system (LMS) which contains study logs, 
achievement rate reports on student-set goals, the identification of students’ weak points, 
and advice on test-taking strategies. CDS, the specific behaviors that an examinee at a 
certain score level can be expected to be able to perform in English (Chauncey Group 
International, 2000) have been conventionally used as descriptors to explain test scores 
to educators and employers who use TOEIC scores to assess the English proficiency 
levels of students and prospective employees. Official CDS published for the TOEIC 
600-score level, and the CDS, at comparable levels, for the Society for Testing English 
Proficiency test (STEP), and for the Common European Framework of Reference for 
Languages (CEFR) were used to compile this study’s CDS set. The CDS were given 
to 592 students as a self-report inventory immediately after they took a TOEIC test. 
The CDS were then used to develop TOEIC learning materials which were trialed in a 
TOEIC skill-building course for 22 students. According to comparisons of pre- and post¬ 
course TOEIC scores, it was concluded that the use of CDS-based TOEIC learning tasks, 
which include advice on test-taking strategies, in a www-based course may help students 
improve their TOEIC scores. 
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1. Introduction 

In 2008, about 778,000 people in Japan took the TOEIC Listening and Reading Tests, 
administered for the creator of the test, Educational Testing Service, by a public- 
interest NPO, The Institute for International Business Communication (IIBC, 2011), 
at official test centers; and many more people took a special institutional version of the 
test at schools and companies (McCrostie, 2009). Because of its widespread use in the 
business community, today virtually every university student who hopes to get a job 
in which English proficiency is a requirement wishes to present evidence to company 
recruiters of a good TOEIC score. A ‘good’ TOEIC score has been defined by individual 
companies; and, certain levels of TOEIC scores are required for initial employment, 
promotions, and foreign assignments. It is believed, according to Terauchi, Koike, and 
Takada (n.d.), that people must have a score of more than 800 to be able to conduct 
international business negotiations in English; and thus, a TOEIC score of 800 is a 
common target for university students. However, the average TOEIC score of first-year 
university students in Japan is 419 (IIBC, 2011); and so, in order to prepare students 
for their careers, TOEIC test preparation courses and intensive seminars have become 
a standard part of the curriculum at many universities. 

In order to help university students improve their TOEIC scores, the authors of 
this paper developed TOEIC learning materials based on can-do statements. In foreign 
language education, CDS have been defined as “descriptions of the competence of 
an individual language user” (Imig & O’Dwyer, 2010, p. 2) and used as a means of 
clarifying plans to improve general EFL teaching practices (e.g., Hiromori, 2009; 
Yamanishi & Hiromori, 2008). Anew set of CDS was compiled by using CDS from three 
sources: TOEIC CDS at the 600-score level with a more or less balanced proficiency in 
the listening and reading sections of the test (Chauncey Group International, 2000), and 
the CDS, at comparable levels, for the Society for Testing English Proficiency test - 
STEP - (Society for Testing English Proficiency, 2006), and for the Common European 
Framework of Reference for Languages - CEFR - (Council of Europe, 2001). 

The 600-score level was chosen because it was approximately mid-way between the 
average first-year university student’s TOEIC score and the common fourth-year target 
of 800. A 600-score was seen as a reasonable target for most students after one or two 
years of EFL study. STEP and CEFR CDS were reviewed in the compilation process 
to make links with many students’ past experiences with English-language proficiency 
testing through STEP tests which are commonly used in Japan from beginning to 
advanced levels of EFL study to measure progress and also with the CEFR standards 
used as benchmarks by international businesses in Japan. Experienced EFL teachers 
made the CDS selections intuitively based on their knowledge of students’ learning 
needs. 

The new set of CDS after confirmation of their recognition by EFL students as 
descriptors of TOEIC 600-level performance abilities were used to develop TOEIC 
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learning materials. The materials have been published in ATR CALL BRIX, a www- 
based courseware in order to make the learning materials available to teachers for 
in-class coursework and outside-of-class assignments, and also to students to use 
independently of coursework through the use of computers with Internet access. 
To enhance independent learning, the courseware includes a learning management 
system which contains study logs, reports of achievement rates for student-set goals, 
identification of students’ weak points, and advice on test-taking strategies. 

The ATR CALL BRIX courseware was designed to create interoperability 
(Ishikawa, Kondo, & Smith, 2010) which is described in Figure 1. Interoperability, in 
this case, is the capacity of the ATR CALL BRIX courseware to create a single learning 
environment in which university students who are trying to improve their TOEIC scores 
and EFL teachers can collaborate in a learning enterprise. The learning environment is 
one of teachers and students in a partnership in which teachers can be said to be doing 
their work most effectively as the students gain independence and control over their 
own learning. 

Figure 1. Design principles of ATR CALL BRIX 



The TOEIC learning materials for this study consisted of 1,000 practice questions 
based on CDS, i.e., practice questions which could be supported by the teaching of 
English-language performance skills. If students completed the TOEIC learning tasks 
incorrectly, the LMS was activated and advice to overcome weaknesses in listening and 
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reading abilities appeared on the display screen. Test-taking strategies for improving 
students’ test-wiseness were also developed and included with the learning materials. 
Test-wiseness is defined as a capacity to utilize the characteristics and format of 
the test and the test-taking situation to achieve success (Millman, Bishop, & Ebel, 
1965; Samacki, 1979). Test-wiseness training can be effective only under specific 
circumstances (Dolly & Williams, 1986), a learning environment which was created by 
the repeated use of ATR CALL BRIX system. Figure 2 shows a conceptual diagram of 
the relationship between the TOEIC learning materials and the LMS in the ATR CALL 
BRIX courseware. 


Figure 2. TOEIC learning materials and the LMS in the ATR CALL BRIX 


courseware 


ATR CALL BRIX 



2. Research questions 


This study investigated the following two research questions: 1) Would a new set 
of 600-score level CDS based on teachers’ intuitions as to its potential for creating 
effective TOEIC learning materials correspond with the beliefs of students, who 
had received 600-level TOEIC scores, about their own English ‘can-do’ abilities? 
2) Would there be improvements in the scores of students who used TOEIC learning 
materials developed from this study’s CDS? 

3. Investigation into research question No. 1 

3.1. Participants 

592 EFL students from first to third years at Kyoto University of Foreign Studies in 
Japan, who took the TOEIC test in January, participated in this study. 
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3.2. Method 

In order to answer research question No. 1, the 592 students completed a self-report 
inventory of the items on the new CDS immediately after they had taken the TOEIC 
test. On a Likert scale of 1 to 5 the students indicated to what degree they had the 
English performance skills described by the CDS. 

3.3. Results and discussion 

Pearson’s correlation analysis showed that there was a strong positive correlation 
between the CDS and students’ TOEIC scores. Students who had scores of 
approximately 600 reported that they had the performance skills in English described 
by the study’s CDS. Students who had lower scores reported that they were lacking 
in the CDS performance skills. Thus, the answer to research question 1 was that the 
new set of CDS corresponded to the beliefs students at the TOEIC 600-level had about 
their own English-language performance skills. It was concluded that because of the 
correspondence between the 600-score level students’ beliefs about their performance 
skills, the lack of these skills as perceived by lower level students, and the EFL 
teachers views of a potential constructive relationship between the new CDS and 
learning materials, the new CDS set would be useful in the development of TOEIC 
learning materials for students who had lower scores but hoped to reach the 600-score 
level. Table 1 shows the correlation among the CDS and students’ TOEIC scores. 

Table 1. Correlation among the CDS and participants’ TOEIC scores 

4. Investigation into research question No. 2 


Measure 

Mean 

TOEIC 

TOEIC 

Can-do 

Can-do 

m 

listening score 

reading score 

listening task 

reading task 

TOEIC 

listening score 

343.2 

(71.95) 

1.00 

.69* 

.45* 

.38* 

TOEIC 

reading score 

267.00 

(68.54) 

.69* 

1.00 

.30* 

.39* 

Can-do 
listening task 

47.84 

(7.61) 

.45* 

.30* 

1.00 

.73* 

Can-do 
reading task 

60.97 

g-36) 

.38* 

.39* 

.73* 

1.00 


N = 592\ *p<001 


4.1. Participants 

22 EFL students at Kyoto University of Foreign Studies in Japan who had TOEIC 
scores of below 600 participated in this study. 

4.2. Method 

In order to answer research question 2, TOEIC testing was conducted at the beginning 
of April before the semester started to select 22 students who had TOEIC scores below 
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600 for a TOEIC skill-building course. The participants used the TOEIC learning 
materials, which were developed based on this study’s CDS set, both in class and 
outside of class. At the end of the course in July, the 22 students took the TOEIC test 
a second time. 

4.3. Results and discussion 

A two-tailed UTest was used to measure the participants’ listening and reading abilities 
at the end of the course. According to the results of the test, the participants’ listening 
ability had improved significantly (p < .01, r = .83), and the participants’ reading 
ability had also improved significantly (p < .01, r = .53). It was concluded that the 
participants’ listening and reading abilities may have improved through the use of the 
TOEIC learning materials. Table 2 shows the results of the total scores of the TOEIC 
test, and the scores for both the listening and reading sections. 


Table 2. Results of pre- and post-TOEIC testing 



N 

Pre-TOEIC Test 

Post-TOEIC Test 


Gam 



Mean 

SD 

Mean 

SD 

Mean 

SD 

Total 

22 

413.41 

82.15 

513.18 

81.42 

99.77 

68.31 

Listening 

22 

247.95 

54.76 

302.27 

49.83 

54.32 

38.18 

Reading 

22 

165.45 

37.45 

201.91 

64.75 

36.45 

58.48 


5. Conclusion 

The study’s two research questions were answered in the affirmative: 

• The items on the CDS compiled for this study were confirmed by students at the 
TOEIC 600-score level to be English-language performance skills they believed 
they had; 

• The TOEIC learning materials which were developed based on the study’s CDS 
was useful in helping students improve their scores. 

The next steps will be 1) to verify that the TOEIC learning materials in the ATR CALL 
BRIX courseware are effective in improving students’ TOEIC scores by conducting 
a study with experimental and control groups, and 2) to identify which test-taking 
strategies in the TOEIC learning materials were used by students at various score levels 
in order to provide better advice in the LMS of the ATR CALL BRIX courseware. 
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